Unsupervised Learning

What is unsupervised learning and how does it relate to the well founded theory of supervised learning? These questions have been discussed during this seminar which brought together neural modellers, statisticians, computational learning theorists (\COLT people") and theoretical computer scientists and physicists. The eld of machine learning with its broad range of pattern recognition applications in data mining and knowledge discovery, in information retrieval and in classical areas like speech and image processing, computational linguistics or robotics is confronted with various problems beyond classiication and regression. The search for structure in large data sets requires automatic inference tools which can also provide quality guarantees to validate the results. The discussions after the talks and in special discussion sessions circled around two main issues of unsupervised learning: 1. What does it mean to detect structure in a data set and how can we quantify it? 2. How can we provide guarantees that the detected structure generalizes from one sample set to a second one? It is unrealistic to expect a general answer to the rst question. A general theory of structure has not been developed yet and attempt like the inference on the basis of Kolmogorov complexity are debated. One might even argue that such a goal is completely elusive since it encompass the program of natural science, engineering and the humanities. The diierent talks, therefore, covered a wide spectrum of special suggestions how structure could be deened and detected ranging from trees in image analysis, informative projections like PCA or ICA representations of high dimensional data, clusters in vectorial data and histograms as well as groups in relational data or principal surfaces. It became apparent in the discussion of simulation results that uctuations in the data should have little innuence on the learned structures. This requirement might be enforced by bounding techniques as they have been developed for the computational learning theory of supervised learning or by information theoretic compression ideas. The challenges of unsupervised learning for the COLT and 1 the modeling community seem to crystallize around the questions how optimally generalizing structures in data can be discovered and how they are characterized and validated in terms of robustness, compactness (description length) and eeciency for learning. What does biology teach us about unsupervised learning? Apart from the miracle how supervised learning might be organized in the brain at the neuronal level, the biological substrate seems to support unsupervised learning …